Countering the Problem of Oscillations in Bat-BP Gradient Trajectory by Using Momentum

Metaheuristic techniques have been recently used to counter the problems like slow convergence to global minima and network stagnancy in backpropagation neural network (BPNN) algorithm. Previously, a meta-heuristic search algorithm called Bat was proposed to train BPNN to achieve fast convergence in the neural network. Although, Bat-BP algorithm achieved fast convergence but it had a problem of oscillations in the gradient path, which can lead to sub-optimal solutions. In-order to remove oscillations in the BAT-BP algorithm, this paper proposed the addition of momentum coefficient to the weights update in the Bat-BP algorithm. The performance of the modified Bat-BP algorithm is compared with simple Bat-BP algorithm on XOR and OR datasets. The simulation results show that the convergence rate to global minimum in modified BatBP is highly enhanced and the oscillations are greatly reduced in the gradient path.